For the Lexicon that Has Everything

This paper argues that it is impossible to separate lexical and encyclopedic knowledge and describes an attempt to build a large lexical database that contains the range of information needed to make a parser or a text generation system interpret and use words and phrases correctly. 1 I n t r o d u c t i o n Any natural language processing system needs both knowledge about words and knowledge about the world. Many natural language systems divide these two kinds of knowledge into two knowledge bases, which we call the lexicon and the encyclopedia for the purposes of this discussion. We argue that the distinction between the lexicon and the encyclopedia is difficult to maintain both in theory and in practice. We describe the design and development of a large lexical database intended to support parsing, generation, and information retrieval applications. We claim that these applications require information of many different kinds, some of which is traditionally stored in a dictionary, some in a thesaurus, and some in an encyclopedia. We need to support not just alphabetic access to this information but access through semantic links. Bierman [1964] was one of the first to describe lexical-semantic links between words. They define the basic organization of semantic information, he claims. He paints an image of a very large single-page dictionary with language-specific nodes connected by semantic relations. Can we distinguish between the lexicon and the encyclopedia in this context? Bierwisch and Kiefer [1970] assume that both kinds of information are contained in the same lexical entry. The distinction between linguistic or lexical and encyclopedic knowledge, they say, corresponds to the difference between the core and the periphery of a lexical entry, where: The core of a lexical reading comprises all and only those semantic specifications that determine, roughly speaking, its place within the system of dictionary entries, i.e., delimit it from other (non-synonymous) entries. The periphery consists of those semantic specifications which could be removed from its reading without changing its relation to other lexical readings within the same grammar. [Bierwisch and Kiefer 1970, 69-70] The major difficulty with this criterion is its instability. As new entries are added to the system, information sufficient to distinguish one entry from another may have to be